A small computer may be used to decode television display material that has been encoded in an economical format (e.g., to permit the transmission of image data via telephone lines or the recording of image data on compact disc). This small computer may be provided with general-purpose memory, portions of which are available for use as image memory to provide buffering between an irregular flow of received image data and the regular flow of image data to the display. It is desirable to provide an image memory configuration that is well suited to being used interchangeably with other data storage in general-purpose memory and does not require the use of dedicated portions of the memory for image storage.
The encoding of television information for transmission over media of such limited bandwidths as those available from a telephone line or compact disc forces the designer to resort to powerful video compression methods. These methods rely upon transmitting as little new image information per frame as possible and upon storing as much old image information as possible; and transmission of new image information cannot be done, at least not entirely, in real time. In order to write a display in real time, then, it is essential then to have frame buffer storage memory with the capability of storing at least two frames of video information. Such memory can be written to from a flow of compressed image data received in non-real time and read from so as to supply the display apparatus with a regular flow of image data in real time. The frame buffer storage memory is bit-map-organized for convenience in constructing updated images from previous image data in accordance with instructions included in the compressed video data.
In present-day practical terms such a frame buffer storage memory is a large amount of memory. Sampling chrominance information more sparsely in space than luminance information can substantially decrease the amount of information to be stored. E.g., where chrominance is sampled one quarter as densely as luminance in the directions of line trace and of line advance, a sixteen times reduction in the amount of chrominance information to be encoded results. If chrominance is described in terms of two orthogonal color-difference signals each having the same number of bits resolution as luminance, which is commonly the case, the amount of chrominance information to be stored in the frame buffer storage memory is reduced from twice the amount of luminance information to be encoded to only one-eighth the amount of luminance information to be encoded.
Image memories, the addressable storage locations of which map corresponding picture elements or "pixels" on a display screen and which store single bits descriptive of whether those corresponding pixels are bright or dark, have been described as being "bit-map-organized" for many years. In recent years the term "bit-map-organized" has been applied to certain image memories in which a pixel variable related to brightness is not expressed in terms of a single bit, but rather in terms of a plurality of bits. Such brightness-related variables may be luminance variables or may be color-difference variables used in connection with describing color displays, for example. The term "bit-map-organized" has been extended to refer to two different memory configurations, each storing a plural-bit value descriptive of a pixel variable.
A plural-bit-variable bit-map-organized image memory of a first general type known in the prior art can be thought of as having employing a number of planes, which number equals the total number of bits in the plural-bit-variable(s) describing a single pixel. The most significant bits of a first of the pixel variables are stored in the first bit plane at storage locations having respective addresses mapping respective pixel locations in the display; the next most significant bits of the first pixel variable are stored in the second bit plane at storage locations having respective addresses mapping respective pixel locations in the display in a manner corresponding to the mapping of the storage locations in the first bit plane; and so forth, proceeding to less significant bits in the first pixel variable, then proceeding through the bits of each other pixel variable (if any) proceeding from most significant to least significant bit. Responsive to a single address this type of memory furnishes simultaneously the respective plural bits of all the pixel variables descriptive of a particular pixel. Essentially, the spatial positions of individual pixels in the display have a one-to-one correspondence with respective image memory addresses, in a spatial mapping. This spatial mapping is held together by the tracing of the display screen and scanning of image memory addresses each being done in accordance with a prescribed pattern of correspondence between these activities. So long as the pattern of correspondence between these activities is adhered to, the rate at which and order in which these activities are carried out do not affect the spatial mapping between the image memory addresses and the spatial positions of display pixels.
A second general type of plural-bit-variable bit-map-organized image memory known to the prior art does not require a one-to-one correspondence between image memory address and the spatial positions of display pixels. There is a list of the values of the plural-bit pixel variables in a prescribed cyclic order, which cycles are arranged in the sequence of the tracing of the spatial positions of pixels in the display. The list is converted to a string of values of the pixel variables, with the bits in each value arranged in prescribed order according to relative significance. Each string of values is divided into words of given bit length, which words are stored respectively in successively addressed locations in the image memory. An image memory of this second general type has to be read out to a formatter with pixel unwrapping capability. The formatter reconstitutes the words into a string of values which are then parsed back into successive values of each pixel variable. The variables for each pixel are temporally aligned by the formatter to be available at the time the spatial position of that pixel is reached in the scanning of the display screen.
When a pixel is described in terms of plural variables--e.g., a luminance variable and two chrominance variables--it has been a general practice to group these variables in a prescribed order for each pixel and to use each group as subvariable components of a respective value of a complex pixel-descriptive variable, when the second general type of bit-map organization is employed in image memory applications. The values of this complex variable are then stored in a bit-map-organized image memory organized as either the first or the second type of image memory described above. This practice is reasonably satisfactory as long as the pixel-descriptive variables used as subvariable components of the complex variable are sampled at corresponding points in display space and with the same sampling density. However, it is desirable to be able to sample the pixel variables at differing sampling densities in order to conserve image memory and to permit faster image processing. Then, this method of using complex pixel-descriptive variables becomes unattractive.
J. A. Weisbecker and P. K. Baltzer in U.S. Pat. No. 4,206,457 issued June 3, 1980 and entitled "COLOR DISPLAY USING AUXILIARY MEMORY FOR COLO INFORMATION" describe an image memory comprising a luminance-only memory, the read addresses of which map display space according to a densely sampled bit-map organization, and a chrominance-only memory, the read addresses of which map display space according to a sparsely sampled bit-map organization. Separate memories, which they refer to as "data memory" and as "small auxiliary memory", are dedicated to the storage respectively of luminance-only information and of chrominance-only information. The read addresses for the auxiliary memory are the more significant bits of the read addresses for the data memory in a scheme for accessing the memories in parallel during reading out from image memory. The Weisbecker and Baltzer configuration of image memory is a variant of the first general type of plural-bit-variable bit-map-organized memory, it is pointed out.
The Weisbecker and Baltzer memory architecture dedicates specific portions of a combined image memory to luminance and dedicates other specific portions to chrominance. Video image storage systems are known where chrominance subsampled respective to luminance for storage in digital memory is spatially interpolated to generate re-sampled chrominance of the same sampling density as luminance, with similar-sample-rate luminance and chrominance signals being linearly combined to generate component-primary-color signals (i.e., red, green and blue signals). Not only can linear interpolation in the direction of scan line extension be used. Bilinear interpolation, where there is linear interpolation both in this direction and in the direction transverse to scan lines, can also be used, for example.
Because of the desire to reduce storage requirements for image memory, which can be accomplished without immediately perceptible degradation of the displayed image by sampling chrominance less densely than luminance, particularly if the image is camera-originated, there is a strong impetus for the designer to configure image memory along the lines suggested by Weisbecker and Baltzer. However, in the Weisbecker and Baltzer configuration of image memory, the number of pixel-descriptive bits associated with an image memory address changes, depending on whether or not a spatial position in the display does or does not have a chrominance value as well as a luminance value associated with it. This interferes with the shifting of bit-map-organized image information in the image memory unless the memory is allowed to have unused bits of storage in it. This, however, undesirably negates to some degree the advantage of sampling chrominance more sparsely in space than luminance. The ability to shift image portions readily in image memory is important in the reconstruction of dynamic images in image memory responsive to compressed video data.
The inventors find it is also unattractive to use complex pixel-descriptive variables in variants of the second general type of plural-bit-variable bit-map-organized memory that subsample chrominance as compared to luminance. The complex pixel-descriptive variables are intermixed with luminance-only pixel descriptive variables in the image memory read out. This presents complex data-parsing problems, especially when shifting of image portions in memory takes place in the decoding of compressed video data.
A type of dual-ported, dynamic random-access memory that has recently become commercially available is the so called "video random-access memory" or "VRAM". This dynamic memory, in addition to a random-access input/output port through which information can be written into or read out of the memory, has a serial-access port from which a row of data can be read serially at video scan rates. The row busses of a principal dynamic random-access memory portion of a VRAM are arranged to transfer data in parallel to a smaller auxiliary memory of the VRAM, during an interval equal to the read interval from the random-access port. A counter is provided in each VRAM for scanning the addresses of the auxiliary memory during its reading, so the auxiliary memory can function as a shift register. After parallel loading of the auxiliary memory, its contents are read out serially through the VRAM serial output port, with the counter counting at a relatively high clock rate. This clock rate can be the rate at which the luminance-only picture elements are delivered to the display monitor of the computer apparatus, for example. This speed of reading is possible because the capacitance-to-substrate of the auxiliary memory busses is relatively low owing to the smaller size of this auxiliary memory. It is attractive, then, the present inventors point out, to use VRAM for the general-purpose memory capable of storing television images, with both the luminance-only information and the chrominance-only information being read out through the serial output port on a time-division-multiplexed basis, although conventional random-access memory can also be used.
In some types of VRAM data can also be serially read into the auxiliary memory via the serial-access port, to be transferred in parallel into the principal dynamic random-access portion of the VRAM. This allows faster writing of the VRAM than is possible by writing information via its random-access port.
Television transmission systems are known where, in order to avoid chrominance information in an analog signal cross-talking with luminance information in an analog signal, lines of chrominance information are time-compressed and are time-interleaved between lines of luminance information. The time compression and time displacement of chrominance is carried out in the digital domain, then transformed to the analog domain by digital-to-analog conversion. These systems are known as "Multiplex Analog Component" transmission systems or "MAC" transmission systems. Luminance/chrominance crosstalk is not a problem in digital television transmission systems such as those considered herein, where luminance samples and chrominance samples are kept separate from each other.
The present inventors discerned that time interleaving of lines of digitized chrominance information with lines of digitized luminance information is usefully applied to the reading of VRAM through its serial access port, in that it permits the use of separate bit-map organizations for luminance and chrominance variables in VRAM. The use of separate bit-map organizations for luminance and chrominance variables the present inventors perceived would avoid the problems encountered in the use of complex pixel-descriptive variables in a unified bit-map organization when chrominance is sampled less densely in display space than luminance is. The use of separate bit-map organizations can be accommodated by using a rate-buffering memory for at least the chrominance samples, the present inventors realized.
In television receivers processing conventional alternate-field line-interlaced television signals to provide progressive scan at doubled horizontal scan rates, a rate-buffering memory is used to receive and delay expanded information, both for luminance and for chrominance. This rate-buffering memory is used for a further purpose, as well, to provide the sample bed information to support spatial interpolation in the direction transverse to line scan. For example, W. N. Hartmeier describes such apparatus in U.S. Pat. No. 4,580,163 issued Apr. 1, 1986 and entitled "PROGRESSIVE SCAN VIDEO PROCESSOR HAVING PARALLEL ORGANIZED MEMORIES AND A SINGLE AVERAGING CIRCUIT". Three line storage memories are operated on a cyclic write-one, read-two basis to provide spatial interpolation in the direction transverse to the line scan. The present inventors developed simpler structures using only two line storage memories for providing rate-buffering and spatial interpolation following VRAM read-out in television display systems of the type with which they are concerned.